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(54) Tide: PANORAMIC IMAGING APPARATUS 
(57) Abstract 

A system is 
disclosed for capturing 
super wide-angle 
panoramic images. In 
particular, a two-reflector 
system is disclosed 
which is substantially 
self-correcting in which 
optical aberrations are 
substantially eliminated, 
such as field curvature, 
astigmatism and the 
like. Moreover, the super 
wide-angle panoramic 
imaging apparatus of 
the invention captures 
a super-wide field of 
view from a substantially 
single reference viewpoint. 
The invention provides 
a substantially compact 
viewpoint, while also 
having a substantially flat 
and stigmatic image plane, 
in the context of a super 
wide-angle panoramic 
system. Devices and 
methods for capturing 
panoramic images of super 

wide-angle scenes are . . 
provided. In a particular embodiment of the invention, two reflectors are provided (e.g., one a hyperboloidal mirror, the other a concave 
ellipsoidal or spherical mirror), a relay system (e.g., optics such as a mirror, a lens, a pinhole and the like) and an image sensor (e.g., an 
electronic photo-sensor, a film and the like). 




BNSDOC1D: <WO 0041024A1J_> 



: FOR THE PURPOSES OF INF&RMATION ONLY 
Codes used to identify States party to die PCT on the front pages of pamphlets publishing international applications under the PCT. 



AL 


Albania 


ES 


Spain 


" LS 


Lesotho 


SI 


Slovenia 


AM 


Armenia 


FI 


Finland .... 


LT • 


.Lithuania „, 


SK 


Slovakia 


AT 


Austria 


FR 


France 


LU 


"Luxembourg 


■ ' SN 


Senegal 


AU 


Australia 


GA 


Gabon 


LV 


Latvia 


sz 


Swaziland 


AZ 


Azerbaijan 


GB 


United Kingdom 


MC 


Monaco 


TD 


Chad 


BA 


Bosnia and Herzegovina 


GE 


Georgia 


MD 


Republic of Moldova 


TG 


Togo 


BB 


Barbados 


GH 


Ghana 


MG 


Madagascar 


TJ 


Tajikistan 


BE 


Belgium 


g:n 


Guinea ■ 


MK . 


- The former Yugoslav , 


TM 


\, Turkmenistan 


BF 


Burkina Faso 


GR 


Greece 




Republic of Macedonia 


TR 


Turkey 


BG 


. Bulgaria 


HU 


Hungary 


ML 


Mali 


TT 


Trinidad and Tobago 


BJ 


Benin 


EE 


Ireland T -~ ' ■* ' 


• v MN : 


Mongolia ." " r K\ 


. UA ' 


Ukraine 


BR 


Brazil 


IL 


Israel 


MR 


Mauritania 


UG 


Uganda 


BY 


Belarus 


IS 


Iceland 


MW 


Malawi 


- onus 


United States of America 


CA 


Canada 


IT 


Italy 


MX 


Mexico 


uz 


Uzbekistan 


CF 


Central African Republic 


JP 


Japan 


NE 


Niger 


VN 


Viet Nam 


CG 


Congo 


KE 


Kenya 1 


NL 


Netherlands 


YU 


Yugoslavia 


CH 


Switzerland 


KG 


Kyrgyzstan 


NO 


Norway 


zw 


Zimbabwe 


a 


C6te d'lvoiie 


KF 


Democratic People's 


NZ 


N sw Zealand . 






CM 


Cameroon 




Republic of Korea 


PL 


Poland 






CN 


China 


KR 


Republic of Korea 


PT 


Portugal.. 






CU 


Cuba 


KZ 


Kazakstan 


RO 


Romania 






CZ 


Czech Republic 


LC 


Saint Lucia 


RU 


Russian Federation 






DE 


Germany 


U 


Liechtenstein 


SD 


Sudan 






DK 


Denmark 


LK 


Sri Lanka 


SE 


Sweden 






EE 


Estonia 


LR 


Liberia 


SG 


Singapore 







BNSDOCID: <WO 0041024A1 I > 



WO 00/41 024 PCT/US00/0OQ50 



PANORAMIC IMAGING APPARATUS 

1. Field of the Invention 

This invention relates to panoramic image sensing of a super wide-angle field of view, and in 
particular, it relates to such image sensing using a two-mirror subsystem that is substantially self- 
- 5 corrected. The term "panoramic" means a 360° field of view in a horizontal plane while the term 
"super wide-angle" means a 120° or larger field of view in a vertical plane. Preferably, the field of 
view in a vertical plane is greater than about 180°. More preferably, it is greater than about 200°. 
And, for the desired apparatus, most preferably, it is greater than about 260°. 



2. Background of the Invention 

10 A perspective imaging system collects rays of light from the scene that pass through a single 

point of reference and projects them onto a sensing element such as film or a charge coupled device 
(CCD). The single point of reference in the perspective imaging system is known as the viewpoint of 
the system. Yamazawa et al., "Omnidirectional Imaging with Hyperboloidal Projection", IEEE 
International Conference on Robotics and Automation, 1993, by Nalwa, "A True Omnidirectional 

15 Viewer", ATT Bell Laboratories Technical Memorandum, BL01 15500-9601 15-01, Jan. 1996 and by 
Nayar, "Omnidirectional Video Camera", DARPA Image Understanding Workshop, May 1997, all 
incorporated herein by reference, describe the need for a single viewpoint. We have, determined that 
the nature of light propagation through the imaging system and the shape of imaging sensor may 
introduce.. geometric transformations in. the Jmage^projected .onto the sensing . element.. Jn a large 

20 number of applications including ^surveillance, remote, sensing, navigation, model acquisition, virtual 
.r^lity, .^ompute^ vision and robotics,, it is de^rabl? tit^^tbese jBra^netoip jtauurfpnnations be corrected 
for the purposes of viewing and analysis. The lack of a single viewpoint introduces aberrations in 
pupils which manifests itself as uncojrectabile geometric transformations (distortions). 

_i c:>, -Vi 'J .:. 1. • :*■: ... r. 

ri' ' > ,1 T ■ • . - . vs.,... 

25 • ^ A classification of imaging systems based on their field of view is: . 

1 . Traditional imaging systems.that image a narrow field of yiew, usually an octant of the sphere 
of view (up to 90 degrees). r 

2. Panoramic imaging systems.that image a panorama ofjhe scene! The field of view can be 
looked upon as a sphere truncated by two parallel planes ..giving a 360 degree field of view in the 

30 horizontal and a limited field erf view in the vertical. 

. : iLiS::0> - ■!«* .*: 

3. Omnidirectional imaging*-, systems that image substantially spherical or substantially 
hemispherical fields of view. 
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1; Dioptric systems that use only refractive elements; 

2. Catoptric systems that use only reflective elements; and 

3. Catadioptric systems that use a combination of reflective and refractive elements. 

Perhaps ; the rimplest system- that provides < perspective projection is a pinhole camera. 
5 Traditionally, lenses have been used in place of a pinhole simpjy because of their superior light 
gathering ability. But a lens, however wide angle it may be, is limited to a hemispherical field of view 
while still maintaining a single viewpoint, although it is physically challenging to design such wide- 
angle lenses. 

Lenses that deviate from maintaining a single viewpoint have been described by E. H. Hall et 
10 al., "Omnidirectional Viewing using a Fish Eye Lens", : SPIE Vol. 728 Optics, Illumination and Image 
Sensing for Machine Vision, * 1986, pp. 250, incorporated herein by? reference. Such lenses have been 
known to achieve larger than hemispherical fields of view, up to 280 degrees in the vertical plane. 
However, these so-called fish eye lenses are significantly larger and more complex than conventional 
lenses, and suffer from severe geometric distortions and loss of resolution in the image over the field 
15 of view. Moreover, the lack of a single reference point for the rays of light imaged by the lens 
disqualifies their usage in a large number of applications, described above. US Patent 5,185,667 to 
Zimmerman and US Patent 5,359,363 to Kuban^are descriptions of additional uses of fish eye lenses, 
all incorporated herein by reference. Thus, of the known dioptric systems, those that seek to preserve a 
single viewpoint are limited to a narrow field of view. *"' ' : 

20 Using only reflective elements, catoptric imaging systems are the closest to ideal imaging 

systems. The. lack of refractive elements removes the possibility of chromatic aberrations allowing 
these systems to operate under a wide range of illumination vvavelengths. But the greatest advantage 
of catoptric systems is that the reflective elements can be matched to correct for almost all aberrations 
that plague imaging systems, including field curvature and spherical aberration in pupils. A 

25 disadvantage of such systems is their light gathering ability which requires them to operate with lower 
F-numbers. 

Catadioptric systems have been known to span the entire gamut in terms of field of view, from 
omnidirectional to panoramic to traditional narrow fields of view. The simplest wide-angle 
catadioptric system comprises ; two optical components: a curved non-planar primary reflector placed in 
30 front of a objective lens. The complete class of mirror lens combinations that capture wide-angle 
views while maintaining a single viewpoint has been described by Nayar et al., "Catadioptric Image 
Formation", IEEE International Conference on Computer Vision, Jan. 1998, incorporated herein by 
reference. Further, for a mirror to have a single viewpoint it is necessary that the mirror be a surface of 
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incorporated herein by reference, have shown that t the only two dimensional curves with focal 
properties are conic sections. Hence, the only mirrors that maintain a single viewpoint are conic 
sections of revolution. Further, those that can be realized in practice are paraboloids, hyperboloids and 
ellipsoids. It is worthwhile mentioning here that while the sphere is an intuitive solution it is 
5 impractical because thetfoctfe isat tHe^enter'bf the sphere/ and so is the cone 'for the teason that the 
* focus is'at the apex of th^«ohe; ,v - >r - • " " r - iy ' 4 ' -*-ns 

A catadioptric omnidirectional imaging system has been described in US patent 5,760,826 to 
Shree Nayar, incorporated herein by reference. The system uses a convex paraboloidal mirror, 
telecentric relay objective lens and a standard camera lens which projects an annular image of a 
10 substaiitiaily hehiisph^ from a r singlfe : viewpoint onto a planar sensing device such 

' ' as a GGDr^A 'disstdvahtagie of thfe system is thS^e iise bf an.aspheric surface results in residual field 
" cuivature? TTiis^reveiitsliS'UsSge with-ai low F4number compact system]- > *<r; : *«• \ . 

A more complex catadioptric panoramic imaging system is one that uses two reflecting 
surfaces in conjunction with a relay objective lens. In such a system the primary reflector collects 

15 scene intensity information which is then reflected oft a secondary reflector into the relay objective 

-r.r v j I ^h" ■ r 5L r. * :^ t.c-n;'*-jif > s'^Hi , • /- "i . '0= \' 

lens. 

. For the entire system to have a. single vjewpoii^t, while the primary mirror must have a single 
viewpoint (which js toe viewpoint of the overal! system) it is not necessary for the^ secondary mirror to 
have a single viewpoint. The tools for developing such systems of m jnrors that have an overall single 

20 viewpoint have been described by Cornbleet, "Microwave and Optical Ray Geometry", Published by 
John Wiley and Sons, 1984, incorporated herein by reference, 'ifcian be shownthat a variety of mirror 
pairs, some'with exotic shapes, can be used to construct catadiopfric imaging systems of interest. 
However we have determined that mirrors with complex shapes produce confounding optical 
aberrations? Moreover, even seemingly simple surfaces" such l as qiiairicV (surfaces of revolution of 

25 planar 2nd order algebraic cilices 1 ) can produce complex optical abenations: In our investigation we 
have found that the only quadfics that can form optically acceptable twd rnmtir systems; ire conic 
sections of revolutions, viz. paraboloids, ellipsoids and hyperboloids. 

It is a well-known fact in geometry that, ia property of conic sections (and hence mirrors 
formed from conic sections of revolution) is tfiat rays of light in the direction of the one focus of a 
30 conic section are reflected by the "conic to converge auh'e'otfer focu^of the conic' section. Hence, we 
have determined that for a conic mirror pair to I maintain a single overall viewpoint it is necessary the 
two conies be confocal, i.e., the far focus of the pfimaiy conic mitror cofneides with the near focus of 
the secondary conic mirror (a special case is the parabola, where the common focus is at infinity). 
When the two mirrors are confocal, i.e., the system maintains a single reference viewpoint, the two 
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mirror subsystem corrects for spherical aberration in pupils. . The imaging system is made complete by 
positioning the entrance pupil of the relay objective lens at the. far fociis of the secondary conic. 

Two-mirror panoramic imaging systems have been described at an internet web site 
maintained by Jeffrey Charles' at http://www.versacorp.com, and in US Patent 4,484,801 to Cox. 
5 While these systems are panoramic in nature, none "seek to preserve a single reference viewpoint, 
resulting in severe geometric distortion in the image captured off the primary reflector. This distortion 
manifests itself in the form of spherical aberration in pupils. The complexity of these systems lies in 
their relay objective lenses, which are required to compensate for such severe spherical aberrations. 
These systems also exhibit complex field curvature,; .by. far the most difficuit aberration to correct. As 
10 a result, these systems have high F-numbers resulting, in additional complexity in the relay objective 
lenses/ Jeffrey Charles' system has an F-number that ranges- from 22 toi6J Cox's system contains a 
12-component lens for field curvature correction; • ' . j ■, . 

Ahbtfier example of similar panoramic systems is described in International latent Application 
PCT/US97/09313 by Dnscoll, et. al. The system uses a primary convex paraboloid Veflector and a 
15 secondary* planar reflector "and hence exhibits severe astigmatism ancl field curvature, requiring 
additional astigmatism correction lenses and field flattening lenses. Additionally, the use of strong 
elements in the astigmatism ^correction lens introduces * deleterious 1 amounts of spherical aberration 
(lack of a single reference viewpoint) an 

Another example of a two-mirror catadioptric system is described in US Patent 3,505,465 to 
20 Rees. The system uses a convex hyperbolbidal mirror as the primary reflector ancl a convex spherical 
mirror as the secondary reflector in conjunction with a complex relay objective lens. J the use of two 
convex mirrors causes the system to exhibit severe field! curvature, hence the neeld for a complex relay 
objective lens. 

Another example of a two-mirror system is described in US Patent ^,566,763 to Greguss that 
25 uses two paraboloidal reflective surfaces, a refractive surface and a telecentric objective lens. It can be 
shown that such a confocal mirror subsystem with two paraboloids that seeks to preserve a single 
overall viewpoint must use a perspective objective relay lens/and hence the system described there 
does not preserve a single reference viewpoint. 

US Patent 4,395,093 to Rosendahl and Dicks describes a two mirror system in which the 
30 primary reflector is a convex-jhyperboloid.and the secondary, reflector is .a concave hyperboloid in 
conjunction with a complex objective lens that comprises 21 components. Since the two reflectors 
have very different radii of curvatures, the system exhibits severe field curvature, which is corrected in 
part by the complex objective lens. 
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an ellipsoid, sphere or oblate spheroid. The secondary reflector is a .convex conic of - revolution, 
typically a hyperboloid althoughtspheres have been used too.: Since the primary mirror is concave, the 
field of view in the vertical, plane is limited to 180°. An^additional 10° im possible by adding a 
refractive negative shell.in front of the entire arrangement. Investigation has shown that no effort was 
5 apparently made to make the two mirrors confocal and further, it is evident that the use of a sphere or 
oblate spheroid in conjunction with a h^erboloid ? results of pupils. 

Hence the system does ( npt have a single viewpoint^ indicating the need for a complex relay objective 

. . , :,US Patent:5;627;675 to Davis ef al. ^discloses a panoramic imaging system that employs as its 
10 primary collection subsystemntwo;elements ithat\comprise a Mersenne optic? the-firsl<jelement being a 
, . convex- paraboloidal. reflector and^theisecond element being* concaye<paraboloidaI reflector- -From the 
above discussion, such a system of paraboloidahmirrors comprises* a; confocal pajr : of Atonic section 
mirrors which preserves a single, reference viewpoint, thus eliminating to a large degree aberration in 

oupils. However, for any field correction to occur in the^two mirror subsystem,, the fcyo mirrors must 

,,*Vv,;v:fi .» r-»io.s •:♦ /tie*,: .>,rr;q 5 ?^cVJ vnc.^/r ;.s Jio^r i « ■ <? t5? .V 

1 5 have the same radii of curvature. Doing so severely, restricts the field of view due to vignetting of the 
scene by the secondary reflector, typically to. less than 45 degrees above and below the horizon. 

v . .v.To r summarize;vithe : ;catadioptriC"rpanomm in the art. all have 

shortcomings. Most systems domot;haye-a*single,^ This 
results in severe spherical aberration in pupils that manifests itself as uncorrectable geometric 

20 distortion in the captured image. Tho$e systems that. do maintain a single viewpoint are limited either 

■ .■*.: . r t'*i • , ; /,aj <.:•• -c'v.iH U;:r"!t "isa/'i .«:;•.> '?"<':■; " > : \ i ' ; 

in their, ability to perform under varying light conditions and F-iuimbers due to significant field 
curvature that remains uncorrected, or in their vertical field of view due to vignetting by optical 
components. Further, the use of objective lenses for correcting above mentioned spherical aberration 
and field curvature results in chromatic aberrations. Furthermore, these objective lenses tend to be 
25 highly complex and expensive. 

The shortcomings of the above-mentioned state of the art are substantially remedied by the 
invention disclosed here. The patents and publications referenced above do not teach the panoramic 
imaging apparatus and method disclosed here. 

3.' Summary, of the Invention . \\ • ^ 

30 One aspect of the invention' is a panoramic imaging apparatus with a super wide field of view 

for imaging a scene, comprising: - 

a. an image sensor positioned to receive said super wide field of view; 

b. a two reflector sub-system that is substantially self corrected, said subsystem comprising: 
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ii, a secondary truncated reflector optically coupled with said primary reflector, said 
secondary truncated reflector positioned to reflect said /.mage reflected by said, primary 
reflector; 

wherein said primary and secondary reflectors have shapes and sizes to substantially correct field 
5 curvature of the image projected on said image sensor, and 

c. a relay system, optically coupled to said secondary reflector, for substantially filtering out 
principle rays that are not reflected by said secondary reflector. 

In an exemplary embodiment, the two mirrors have shapes and sizes to capture the super 
■ : wide-angle scene from a single reference viewpoint. 

10 A super wide-angle field of view, for the purpose of this invention, comprises a field 6f view 

greater than or equal to 120 degrees. That is, it constitutes a cone which extends at least 60 degrees 
from the optical axis. -There may exist a blind ispot in the immediate vicinity of the optical axis. 
Preferably, the super wide-angle fieki of view of the present invention includes a panoramic field of 
view greater than ; or equal to .180 degrees, more preferably greater than or equal to 200 degrees. In 

1 5 selected embodiments of the invention, the field of view may, be greater than or equal to 220 degrees. 

In an exemplary embodiment of an ^apparatus .according to the present invention; the primary, 
reflector is convex. In another exemplary arrangement, the primary reflector is concave. , 

In yet. another exemplar embodiment, the surface of v t!ie primary reflector is a conic of 
revolution that obeys the following equation, ?*"•;»;■. 

20 r 2 =2R y z-{\ + k x )z 2 . (t) 

Since the reflector has axial symmetry, equation (1) also : represents a cross section of the 
reflector, r is the radial coordinate, i.e., r^ + y2 and zjs the coordinate T along the optical axis Z. 
R\ is the radius, of curvature of the conic that forms the reflector and, k\ is its conic constant. For it] <- 
1 , the reflector is a hyperboloid of revolution, for k\ = r 1 » a paraboloid, for - 1 <k \ <JQ. Note that for the 
25 special, case of k\= 0, the primary reflector is a sphere. 

The reflector is truncated in a plane that is perpendicular to the optical axis, Z; such that the 
. desired field of view is imaged by the reflector. 

A hole, having a selected diameter, is created at the vertex (apex) of the reflector to create an 
optical path through which light, reflected off the secondary reflector can pass .without substantial 
30 vignetting. The hole serves the additional purpose of discarding rays of light that are reflected by the 
primary reflector such that they do not substantially pass through the single viewpoint of the imaging 
system. 

In another exemplary arrangement, the primary reflector is a higher order (>2, e.g., 14) surface 



BNSDOCID: <WO 0041024A1_I_> 



WO 00/41 024 PCT/USCO/00050 

7 

. In an exemplary embodiment of an apparatus according -to the present invention, the secondary 
reflector is concave. In an alternative arrangement, the secondary reflector is convex. 

In yet another exemplary embodiment, the secondary reflector is in the form of a conic of 
revolution. The secondary reflector is preferably positionVd along an optical axis, which coincides 
5 with the optical axis, Z, of the primary reflector. The surface of the secondary reflector obeys the 
following equation, 

r 2 =2R 2 z-(\+k 2 )z 2 ,^ . r:t ;: . ; . ... , . : (2). ... n- . 

where once again, due to axial symmetry, equation (2) also represents a cross section of the reflector 
wherein r. is the radial coordinate and z is the coordinate along the optical, suds, Z R2 is the radius of 
10, - curvature of the t conic that [forms the secondary reflector and^ »s its conic constant. For *2<-V, the 
. reflector isa-hyp^rboloid of revolution, for k$r -1. a paraboloid, for .-J <*2^ an ^ellipsoid*: . Nqte that 
for the special cas.e ofi*2 = °> the secondary, reflector is ; .a sphere^ . ■,. -r -Kr. t 

the reflect©? is' truncated -in a-planc that is perpendicular to the 'optical axis, 1 Z; such that light 
reflected offahe primary reflecior is imaged by the secondary reflector. r; ' ■ 

15 "'^ In another exemplary arrangement^ the secondaiy reflector is a .higheV order (>2, e.g., 14) 
surface of revolutibrir- ^«" s ^ ■* ' • i * r " *+* • 1 ' - ^ • -^o-"^ * J » 

In an exemplary embodimeht^ the distance between thei^apexes^df the. two reflectors 
substantially obeys the following equation: » " 

_ 2R,^k x | jg 2 fl, (3) 

20 x lh an exemplary embodiment,* th^ relay 4 system is preferably' positioned along an optical axis, 

which coincides with the comWori opticil 'tikiifZ, 'of the prtmaiy ^{(''isecond^'i^flecforS; such that 
the "primary reflector is located physically between the relay systehi'ahH the secondaty reflection The 
relay system is an optical component used to project ah image 1 to another' location, he.; ft is a means for 
relaying the image to the sensor. :/ - »v 

25 In another exemplary arrangement, the relay System is a simple aperture/ as in a pin-hole 

• -camera. . ■ . ' • . 1 " ' r. .^.:!?-( ;r , ! * . 

In another exemplary arrangement, the relay system comprises at least one lens. The lens may 
be made of a plurality of portions of optical miateHaT Aat P are ! c6mentecl togetheH* such as a doublet. 

In an exemplary embodiment, the distance between the vertex of the primary reflector and the 
30 entrance pupil of the relay system substantially obeys the equation: 
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In an exemplary arrangement, the image sensor is electronic, such as a charge coupled device 
(CCD) or a complementary metal oxide ; semiconductor sensor (CMOS) and provides an electronic 
signal that is representative of the image projected onto the image sensor. This image signal is 

5 digitized and; transferred to an image storage apparatus. The digitized image can: then be transferred to 
an image processing apparatus. In another exemplary arrangement, the digitized, signal is transferred 
directly to an image processing apparatus, without.using an intermediate storage .apparatus. In yet 
another exemplary arrangement, the image sensor is photographic film, the image of which can be 
subsequently digitized and the resulting signal then transferred to the imaging processing apparatus. 

10 The imaging processing apparatus is advantageously adapted to enable viewing of any portion of the 
super wide-angle scene. 

Another aspect of the present invention is a two mirror system that substantially eliminates to a 
very large extent, aberrations tHat occur in panoramic optical systems, allowing for the creation of an 
image of high optical qudity/said system 1 comprising: 

15 a. a primary reflector that constitutes the first mirror, and 

b. a secondary reflector that constitutes' the second mirror, the shape and size of which is 
carefully matched to the shape and size of said primary reflector. 

To minimize field curvature in an optical system, the Petzval curvature of the system must be 
close to zero. While this corrects third order field curvature, higher orders of the field curvature can be 
20 corrected by optimizing optical powers of the optical components. The result is a diffraction-limited 
system with highest possible image quality. > 

Yet another aspect of the present inviention is a two mirror subsystem- that substantially reduces 
the cost of manufacturing a commercial super wide-angle panoramic imaging apparatus, and* increases 
versatility of use, said subsystem comprising: 

25 a. a primary reflector that constitutes the first mirror, and v 

.b. a secondary reflector that constitutes the second mirror, the shape and. size of the two mirrors 
chosen to minrnr,; ize field curvature introduced by the two mirror subsystem. .- 

It is known that in systems with field curvature, z. non-trivial array ofjsnses is required to 
offset the negative effects of field curvature, this array increasing the cost of the system. Additionally, 
30 field curvature correcting lenses or field flatteners are typically introduced in close proximity to the 
image sensor, reducing the versatility of the system with respect to use with standard, off-the-shelf 
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Another aspect of the present invention is a panoramic imaging apparatus that senses a super 
wide-angle scene from a single reference viewpoint. . In an exemplary embodiment, the reference 
viewpoint is a locus of points that lie within a sphere of radius of no more than 4 mm. 

The present invention also.proyides a method for sensing an image of super wide-angle scene, 
5 which in .an exemplary embodiment of the present invention, comprises the following steps: ( , 

a) reflecting^an image <tf -the super" wide-angle ffeid< of- vtew on a primary reflector (described 
7 above) SUch that 1 the- single viewpoint substantially'coincides with the nefr focus of the conic 

^ ^section th^formS^ - - 1s- 

b) fVl1 ' reflecting the image reiflected in step (a) on a secondary reflector (described above) such that 
10 F " " ,l L the S rays^ reflector pass through the hole on the primary 

rfeflector, 

c) propagating the rays of light from step (b) through a relay system and 

d) sensing the rays of light propagated through the relay system in step (c). 

The present invention also includes, in another exemplary .embodiment of .the imaging method, 
15 the further steps of: , : „. . 

1 providing an image signal which is representative of the image projected onto the image 

sensor > „ : -^Lr-V *.i to. _ .-™>< . . ■. ■ . 

2. converting the image signal f to image dat^ mapjping the image data into a Cartesian-coordinate 

20 3. interpolating the image date and fonning ardigital imagQ from the mapped image data and the 
interpolated image data. ' l»"-o r 4 

: Anotheraspect of the ; present invention is a.two mirror panoramic systernithat is substantially 
- scalable, in that the.size of all compQnents ; and-their>relatiY0.positions can. be scaled without significant 
loss of image quality. 

25 4. Brief Description of the Drawings" 4 

FIG. i • X sectional view depicting^ first- exemplary arrangement with a primary hyperboloidal 
mirror, a secondary ellipsoidal mirror, a pinhole relay lens, and a 200 degree field of view. 

FIG. 2 Depiction of Field curvature (PIG; 2a) and "distortion graphs (FIG. 2b) for the arrangement 
of FIG. 1. ' ' ' 

30 FIG. 3 Depiction of a Modulation Transfer Function graph for the arrangement of FIG. 1 . 

FIG. 4 A sectional view depicting a second exemplary arrangement with a primary hyperboloidal 
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* FIG. 5 Depiction of Field curvature (FIG. 5a) and distortion, graphs (FIG. 5b) for the arrangement 
of FIG . 4 

FIG. 6 Depiction of a Modulation Transfer Function graph for the arrangement of FIG. 4. 

FIG. 7 A sectional, view depicting a third exemplary arrangement with a primary hyperboloidal 
5 , , mirror, a secondary spherical mirror, a pinhole relay and a 220 degree field of view. 

FIG! 8 Depiction of Field curvatfire (FIG. 8a) and distortion graphs (FIG. 8b) for the arrangement 
• of FIG. 7. ■ ' ; ■- • ■ • 

FIG. 9 Depiction of a Modulation Transfer Function graph for the arrangement of FIG. 7. 

FIG. 10 r A sectional view depicting a fourth exemplary arrangement with a primary hyperboloidal 
10 ..mirror, a secondary je.Uipsoida! mirror, a pinhole relay lens, and a 260 degree field of view. 

FIG. ; ! Depiction of Field curvature (FIG. M s l a) and' distortion graphs- (FIG. 1 lb) for the 
- : arrangement of FIG. 10. - ' - ' f' , . 

FIG. 12 Depiction of a Modulation Transfer Function graph for the arrangement of Tib. 10. 

FIG. 13 A sectional vjew depicting, a fifth exemplary arrangement with a double-Gauss relay 
15 objective lens. 

• FIG. 14 1 A sectional view depicting : a si;;th exemplar}' arrar-gement cf an xopto-rneohanical 
implementation made from a single block of optical material: 

FIG. 15 A sectional view depicting a seventh exemplary arrangement of an opto-mechanical 
implementation that uses a protective transparent dome. 

20 FIG. 16 A sectional view of the path of a principal ray of light in a confocal two-mirror system, in 
accordance with the invention, in which the primary mirror is a hyperboloid and the 
secondary mirror is an ellipsoid. 

FIG. 17. Illustration of r.n optical layout of a Miniaturized Riggedteed Paracamera. 

FIG. 18 * Illustration of a sectional view of the Miniaturized Riiggedized Paracamera with ray 
25 * " tracing. , ^ 

FIG. 19 Illustration of a sectional view of the Miniaturized Ruggedized Paracamera with groove, 
and top and bottom pins to prevent ghpst images and ambient light. 

FIG. 20 Illustrates , an embodiment according to the invention that has a two. mirror system, 
. comprising a primary hyperboloidal truncated reflector ancTa secondary spherical reflector. 

30 FIG. 21 Illustrates an embodiment of an apparatus according' to the invention, having a primary 
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FIG. 22 Illustrates an embodiment having a primary truncated reflector that is a higher order 
ellipsoidal surface of revolution, and having a secondary spherical reflector. 

5. Detailed Description of the Preferred Embodiments 

A panoramic imaging 'apparatus for sensing a super wide-angle fieldof view is provided by the 
5 present invention. The apparatus comprises: (a) a pair of reflectors; including (i) a primary reflector 
having a first shape and a first size and capable of providing a first reflection , of a super wide-angle 
scene, and (ii) a secondary reflector having a second shape and a second size, which is positioned to 
accept the first reflection to provide a second reflection at a first location, the shapes and sizes of the 
primary and secondary reflectors, and their respective positions, are such that the second reflection 

10 provided at'thfe ; firsi location is substantially free of field curvature effects and astigmatic effects; (b) a 
"• relay system' : c'6Wiprising optics Capable 6f relocating the second reflection to a second location to 
provide a relocated reflection; and (c)*n image senior positioned to accept the relocated reflection at 
the second location, in which the panoramic imaging apparatus captures -thpi super wide-angle scene 
from a substantially single references viewpoint. 

15 In a particular embodiment of the invention, the panoramic imaging apparatus is equipped with 

a rday system that includes optics' capabft n df substantially filtering buf optical rays other than those 
reflected directly by the secondary reflector from incoming optical rays Reflected directly by the first 
reflector. ^Moreover, in onc^embsdiment the: second reflection is substantially flat and stigmatic. In a 
preferred embodiment the primary! reflector is convex and the secondary reflector is concave. More 

20 preferably, the primary reflector is substantially hyperboloid and the secondary reflector is 
substantially ellipsoid. 

Thus, an alternative embodiment of the present invention includes a panoramic imaging 
apparatus for sensing a super wide-angle field of view comprising: (a) a pair of reflectors, including (i) 
a primary reflector having a first shape and a first size and capable of providing a first reflection of a 

25 super wide-angle scene, and (ii) a secondary reflector having a second shape arid a second size, which 
is positioned to -accept the first ^reflection tc provide a second reflection at a first, location, the .shapes 
and sizes of the^ primary and secondary reflectors, and their respective positions, are such that the 
second reflection provided at the first location is substantially free of field curvature effects and 
astigmatic effects; (b) a relay system positioned to accept the second reflection and comprising optics 

30 capable of substantially filtering out optical rays other than those reflected directly by the secondary 
reflector from incoming optical rays reflected directly by the first reflector' to provide a filtered 
reflection; and *(c) ah image sehsbr positioned to accept the filtered reflection, in which the panoramic 
imaging apparatus captures the super wide-angle scene-frdm § substantially single reference viewpoint. 
Preferably, the alternative embodiment is equipped with a relay system that includes optics capable of. 

35 relocating a reflection (any reflection, e.g., a second reflection or a filtered reflection) at a first location 
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The image sensor may comprise any suitable sensor, including but not limited-to photographic 
film, a CCD image, sensor; a CMOS image sensor a video camera and the like. Also, the relay system 
is positioned such that the primary reflector is between the secondary reflector and the relay system. 
The relay system may comprise any suitable relay including but not limited to a pinhole, at least one 
5 lens, at least one mirror and the like. . • , 

In a preferred embodiment of the invention the primary reflector comprises a: substantially 
hyperboloidahmirrorihfcving a surface that substantially obeys the equation expressed in cylindrical 
coordinates, r 2 = 2R x z- (1 + Jfc|)z 2 , where z is the coordinate along the optical axis, r is the radial 
coordinate, Rj is the radius of curvature at the vertex of the conic that forms the priniai-y reflector and 

10 kj is the conic constant of the primary reflector, and the secondary reflector comprises a substantially 
ellipsoidal or spherical mirror having a surface which substantially obeys the equation expressed in 
cylindrical coordinates, r 2 =2/? 2 z-(l + k 2 )z 2 , where z is the coordinate along the optical axis, r is the 
radial coordinate/-^ is the radius of curvature at the vertex of the conic that forms the secondary 
reflector and k*i is the conic constant of the secondary reflector.' In addition the primary reflector has 

15 an axis of rotation, comprises a mirror truncated at a plane 1 that is substantially perpendicular to the 
axis of rotation and further has a near focal point and a far focal point. Likewise, the secondary 
reflector has an axis of rotation, "which is substantially coincident with the axis of rotation of the 
primary reflector, comprises a mirror truncated at a plane which is substantially perpendicular to the 
axis of rotation of the secondary reflector and further has a near focal point and a far focal point. 

20 The configuration of the apparatus of tlie invention can be made such that a position of the 

near focal point of the secondary reflector is substantially coincident with a posiiionof the far focal 
point of the primary reflector, and, optionally, the reiay system is positioned along the 'axis of rotation 
of the primary reflector and the secondary reflector. If desired the image sensor is positioned along the 
axis of rotation of the primary reflector and the secondary reflector/ The apparatus may further 

25 comprise a transparent mechanical support, which maintains the relative positions of the primary 
reflector, the secondary reflector, the relay system and the image sensor. In a particular embodiment, 
the image sensor provides an image signal representative of the relocated reflection and which 
apparatus further comprises an image signal processor that is coupled to the image sensor to convert 
the image signal into image signal data and to map the image signal data into a Cartesian coordinate 

30 system. The image signal processor may further include an interpolator for providing interpolated 
image data, such that the interpolated image data and the image signal are combined to form a digital 
image. 

The panoramic imaging apparatus of the invention may be equipped with primary and 
secondary reflectors that include back reflective mirrored surfaces of a solid transparent optical block. 
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.. The instant invention also contemplates a method of providing a substantiially stigmatic 
reflection of a sdper wide-angle field of view captured from a substantially single referfehce viewpoint 
comprising: (a) providing a primary Teflector to generate a first-reflection ofia super; wide-angle scene, 
the primary, reflector having amar focal point and a far focal point; (b).providing a.secondary reflector 

5 to accept the first reflection and to generate a second reflection, the secondary reflector having a near 
focal point and :« far focal- point; .(c) providing a relay system: having an entrance pypil; in which the 
primary reflector, -secondary reflector and lens are;, positioned such^that the; .fair, focal ? point of the 
primary reflector ./coincides substantially with the/ near focal point of\the secondary reflector and the 
entrance p.upil of the relay system^cpincides .substantially .with the far. focal point of the secondary 

10. . reflector^ In ,a preferred embodiment the rays of lightofc the second reflection pass, through a hole in 
the -primary jeflector and . meet subs(antiaily n at a t single gointpn the. optical . axis of. the primary 
reflector. , More preferably the metho^ further comprises substantially filtering out optical rays other 

than those reflected directly by the secondary reflector from incoming optical rays reflected directly by 

■ • . . ->.j 1 • -r-j u \ ■ A *s /*«;::.• to ju^v: A, 2s : J k c p \v:.i ! 

the first reflector to provide a filtered reflection. Even more preferably, the method further comprises 
15 propagating the second reflection or the filtered reflection through the relay system Jo generate a 
relocated reflection and most preferably further comprises sensing the relocated reflection. In 
addition, the preferred method of the invention utilizes a primary reflector and a secondary reflector 
having shapes and sizes that give rise to a second reflection that is substantially free of field curvature 
effects. 

. , r. ii * v \ ;i { >..>n * r.« j ;r-. 1 . ? v • . • r >v ■ ..» . 

20 The present invention also contemplates a method of providing a substantially planar and 

stigmatic reflection of a super wide-angle field of view captured from a substantially single reference 
viewpoint comprising: (a) providing a primary reflector having a^ first shape and a first size to generate 
a first reflection of a super wide-angle scene; and (b) providing a secondary reiflector having a second 

shape and a second size to accept the first reflection and to generate a second reflection, in which the 

; ...:)■ "la ' ■ it ' > f \ cjij "j: . v. 'ii i* \ ^.'i! ' . , s r-, 

25 shapes and sizes of the primary and secondary, reflectors, and their respective positions, are such as to 
give rise to a second reflection that is substantially planar and stigmatic and in which a super wide- 
angle field of view is captured from a substantially single reference viewpoint. In specific 
embodiments the method further comprises substantially filtering out optical rays other than those 
reflected directly by the secondary reflector from incoming optical rays reflected directly by the first 

30 reflector to provide a filtered reflection. In addition, the method may further comprise relaying the 
filtered reflection to another location to provide a relocated reflection and then sensing the relocated 
reflection. 

The teachings of the present invention will now be understood by a person skilled in the art, by 
reference to the following description of the inventive apparatus. Illustrative examples of 
35 embodiments are also provided below as examples, that will be understood by those skilled in the art, 
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as demonstrating how an apparatus according to the teachings of the present invention may be made 
and used. 

Similar reference numbers in the figures refer to corresponding elements in different 
embodiments. * 

5 FIG. 16 shows a sectional view of the path of a principal ray of light in a confocal two-mirror 

system, in which the primary mirror is a hyperboloid and the secondary mirror is an ellipsoid. 
Principal ray of light l\ from the scene in the direction of the virtual entrance pupil (which coincides 

with the near focus Fj of the primary hyperboloid mirror Cjj, strikes the primary mirror Ci and is 

...... i 

reflected along I2 by the hyperboloidal mirror in the direction of its far geometrical focal point Fj, to 
10 intersect the optical axis at F'j. A real intermediate image of the virtual entrance pupil is formed at 
this point. 

. .The reflected ray I2, then strikes the secondary ellipsoidal mirror C2. Since the near 
geometrical focal point F2 of the secondary mirror coincides with F' j, the ray I2 is reflected by the 
ellipsoidal mirror along I3 in the direction of the far geometrical focal point F'2, through the hole in 
15 the mirror C\. A. real intermediate, image of the object space is formed immediately after reflection 
from the ellipsoidal mirror. Note that real intermediate images for the pupil and for the object are 
located in different optical spaces: the former lies in the opticaj space between the two mirrors, while 
the latter lies in the spape between tlie second mirror and the relay lens Rj 

The entrance pupil of the relay objective tens Rjj's located atthe far focus of the ellipsoidal 
20 mirror F'2,' conditionally shown in FIG. 16 as a real diaphragm; The relay objective lens forms a final 
image of the scene on a planar two-dimensional' sensor, surface X The image is annular with a small 
black circle in the center: a result of vignettingof ths object space by the secondary mirror and the 
hole in the first mirror Cj. 



The relationships bstween the various £ng!es in the. figure are given, by; 

(l + jfcj'jsinfl 

2/^c~ + (1^ *i ) cos 



25 . mfi- 41^'> sing , (5) 



tana = — =^= — — — . ' (6) ' 



and 



2J-k 2 + (\-k 2 )cosp 
Equations (5) and (6) give: 

tana = (j+M*"* : , (7) 



27^~*7 + O-* ff )cos0 



Rwsnnnin- <wn nruircuAi i > 



WO 00/41024 PCT/USOO/00050 



15 



(8) 



6. Examples 

6.1. Example One 

FIG. 1 illustrates a first exemplary embodiment of a panoramic imaging apparatus 
5 according to the present invention. A convex hyperboloidal reflector C\ had a 25.658 mm radius of 
curvature at the vertex and a conic constant k| = -3.6205. A concave, ellipsoidal reflector C2 had a 
32.150 mm radius of curvature at the vertex, and a conic constant k2 = -0.2398 was positioned at a 
distance 50 mm from the Cj mirror. A pinhole camera with focal length 7 mm was positioned at a 
distance 13 mm from the vertex, on the right hand side of the C\ mirror, and played a role of 
10 microprojection lens, image plane I was at a distance 8/43 mm from the pinhole lens. The panoramic 
imaging apparatus covered a 2QQ degree vertical field of view and a -360 degree horizontal field of 
view, its F number was 2.5, and focal length was -0.9 mm. Principal rays of the bundles of light 
beams from the panoramic scene went into the hear virtiiai geometrical focal point of the hyperboloid 
mirror CI (not shown in Fig. 1 j, struck the mirror surface, and were reflected by Cf such that they 
15 passed throiigh the far geometncarifdcal^ the hyperbolic mirror C\[ Acohcaveeiiipsbidal 

reflector C2 was positioned siidh that its* near "gebrhetfical focal point F^ ' coincided with Fj. 
Ellipsoidal mirror C2 received the reflected- :mage from CI and reflected jt ( back through a hole in the 
" primary reflector, such Jhat all primary, rays of4ight passed: through, its far foous^F'2,. that coincides 
with an aperture, R, offte, pinhole lens Hj to filter out rays lofilight that do not pass through F'2- An 
2C image sensor I was'positioned . behind the aperture to sense the image.of.the scene. 

FIG. 2(a) is a graph of the field curvature of the two mirror system for this example. As can 
be seen from the graph, the system has a substantially flat panoramic imaging field cf view. Total 
nonflatness is less than 10 micrometers. The optical system optimization was done using conic 
constants and ridii of curvature of both mirrors to correct for higher order field.curvature. Further, the 
25 graph shows that both saggital and tangential field curvatures (depicted by S and T in the figure) 
follow each other. This is indication that the system has no astigmatism in addition to a substantially 
single viewpoint of reference for the entire imaging system. 

FIG. 2(b) is a graph of distortion in the imaging system. In the present case, we consider it a 
specific image compression, which may be appropriately uncompressed by computer software. 

30 FIG. 3 is a graph of the modulation transfer function (MTF) of the two mirror imaging system 

as compared with a diffraction limited system for polychromatic light. As can be seen from the graph, 
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This exemplary embodiment, as do the others described herein, illustrates the two key features 
of the present invention: a substantially compact/single viewpoint of reference and a substantially flat 
and fully stigmatic image over the entire field of view. 

Although, the description herein is with regard to visible light, it is to be understood that the 
5 present invention has equal application to other forms of electromagnetic radiation such as ultraviolet 
light or infrared light, and the appended claims are to encompass an apparatus that works with these 
forms as well as with visible light. 

6.2. Example Two 

FIG. 4 illustrates a second exemplary embodiment of a panoramic imaging apparatus 

10 according to the present invention. A convex hyperboloidal reflector C\ was positioned to reflect an 
image of a 200 degree field of view. Principal rays of light from the scene that are reflected by 
were picked up by the secondary reflector, C2, that was a truncated sphere. The reflection by Cj 
obeyed the laws of Gaussian optics, in F2 and F'2 are the two conjugate optical points of the reflector 
C2. Rays of light that passed through F2 were reflected by C2 such that they passed through F'2. In 

15 this exemplary embodiment, F2 substantially coincided with F'x, the far focus of C\. When the two 
foci do not coincide exactly, the dominating factor in the system is the reflection law by C2. In such 
an arrangement, the imaging system will not have an exact single viewpoint of reference. Instead the 
viewpoint travels on a surface that is known as the caustic of reflection. For positions of F2 that are 
between H| and F'i,.the caustic is a substantially compact. surface and the viewpoint of the entire 

20 system can be approximated to Fj with minimal distortion in the sensed image. Once again, R, was a 
pinhole aperture that filters out rays, that, do not substantially belong to the sensed, image and I was an 
image sensor such as a CCD, CMOS, film, pr pther sensing device adaptable for use with the, optical 
apparatus of the invention. The focal length of the pinhole camera was 6 mm, the back focal length 
was 7.8 mm. its. distance from the. mirror C\ .vertex was 10 mm. The image circle diameter was 4 mm, 

25 the F. number of the system was 2,5, and thefocal len^h : was;1.2 mm. , t , 

FIG. 5(a) shows a graph of the field curvature of the two mirror imaging . system, r As can be 
seen from the graph, the two mirror system has a substantially flat field of curvature, nonflatness is no 
mere than 6 micrometers.* "in addition, the astigmatism exhibited by the system is no more than 2 
micrometers. Spherical aberration in pupils is very small and hence the viewpoint of the entire system 
30 is substantially compact and can be approximated as a single point without any loss in quality. 

FIG. 5(b) shows a graph of the compression in the two mirror system. 

FIG. 6 shows a graph of the modulation transfer function (MTF) of the two mirror imaging 
system as compared vvfth a diffraction limited system for polychromatic light. As can be seen from the 
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indicating that the optical quality of the two, mirror system is close to id k eal. Resolution of the system 
is 400 lp/mm (line pairs per mm) under 40% of contrast • . . ,-ni 

6.3. Example Three 

FIG. 7 illustrates k third exemplary arrangement 6f the present' invention wherein the 
5 primary reflector was hyperbolbidai in shape and the secondary reflector r was spherical, hence 
ellipsoidal. Field of view of thil'system was 220 degrees ma vertical plane, the F number was 2.8, the 
focal length was -0.95 mm, the image diameter was 3.6, which fitsHo a one third inch (0.8$ cm) CCD. 
The microprojection relay lens had a focal length of 6.25 mm, a back focal, length of 8.245 mm, and a 
distance from the C\ vertex of 9 mm. Once again, as in the earlier arrangement, the position of the 
10 secondary reflector was between the far focus of the primary reflector and the apex of the primary 
reflector. In such an arrangement, the spherical aberration in pupils was minimal as can be seen from 
the graphs of saggital and tangential field curvature from FIG. 8(a). The graphs of S and T closely 
follow each other, indicating a compact caustic of viewpoints'. FIG. 8(b) shows a graph of the 
distortion by the system. FIG. 9 is a graph ofthe MTF of the system, which shows that the system 

15 closely follows a diffraction limited system which is the physical limit on an optical system. The first 

:: ■ ■ ;' ".- . .... • » • r» . ±''1 \ u. J .ui! i yj v », - . ..» : j.-'v;- 

mirror diameter was less than about 5 cm. As in the case of the second example, the apparatus of this 

■■ ■ •. -i «, • ... \ ei f.nJ ;:: 'Xfoi j^.-uvir ^ ..J '.'V'-V • % t---**vr * " . 

example fits to commercial robotics and is suited for machine vision applications. 

' - .*■■■• •: »" * - .ii.. TOGO u» v^v/l f\~ .^-^fe sri .V 4 ; ,\ .. 

6.4. -Example Four ^ 1 • - - ^ <• :Ar ! v ' : '" 4 ' ' 1 ' 

FIG.'s 10, H(a)(b) and 12 illustrate ^'fourth exemplary embodiment tltat was similar 

20 to the first Embodiment iif FIG " T. * : TKe 'printer^' bkd l 'secMhdaiy mirrors' Were'^drtcb again' convex 

hyperboloidal and concave ellipsoidal^ respectively- arifr had airhost equal radii : bf 'curvature at the 

vertices. ' Field' of View of 'the system wal 2^0 decrees in 1 a vertical pliiie, fdcaf'leri^Was-^O.e mm, F 

number wks 2.8i ahdlmage 'diahieWr was 3.6 trim. ' Rfelay focal length Was* 10.25 mm; its back focal 

length Was' 13:2 mm. image quality Was cbmjft&flSft (300 fp/mm' uriHe^ 1 46% J 6f contrast), 'as for the 

25 second and third examples. From FigM 1(a) 'dari r b6 'seen the Very hi|h : l&vtt of the astigmatism and 

field curvature correction. . Image.flatniess is in the rangerof.from abput^rfo about 5 roicroftieters. This 

kind.of image quality correction?.was achieved; without introduction, of. higher:orderaspheri<:aI surfaces 

for the two mirrors.. On the other hand,;the^fl^st hyperbplici mirror. diameter was»about 135.5 mm, i.e., 

aboutthree-timestthat ofithe^first-andsecondexamples.Hi: •:.*>. i if^r.-^*. ! . '-sf 1 .^' 

• " ■ . : * : . ' i '** \\' wsytw i '* r :n ' 'no:. .' ' . ; ? • ■ 
30 6.5. Example Five 

FIG. 13 shows a fifth exemplary amhgemeht in which the relay objective was now a 
Double-Gaussian type of lens, indicated by 5. The rest of the arrangement's similar to that in FIG. 1, 
in that the primary mirror was hyperboloidai in shape, the secondary reflector was ellipsoidal in shape 
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and the two conies were strictly co'nfocal. Again, in this arrangement, the caustic of viewpoints was 
substantially a single viewpoint and hence the system had a single viewpoint of reference: "<■' 

6.6. Example Six 

FIG. 14 shows a sixth exemplary arrangement which showed a possible opto- 
5 mechanical arrangement for the two reflectors. In this arrangement, the two reflectors were made of 
the same block of transparent plastic material such as an acrylic resin. The primary reflector is 
indicated by 2 and was formed by shaping one surface of the block as in the shape of a hyperboloid of 
revolution. The outer surface of the block was coated with an appropriate reflecting material, 
transforming the surface 2 to act like a'hyperboloidal back reflector. The surface 3 was shaped to 

10 represent an ellipsoid of revolution or a sphere, as per the arrangements discussed above, and coated on 
the outside with appropriate reflective material to' form a back reflector. The length of the block obeys 
the equations described above and depends on the particular arrangement' 1 of reflectors. The 
appropriate central part 4 of the surface 2 was left uncoated, so that it worked as a refractive surface, 
having the same shape as hyperboloid 2. Surface 1 was transparent to allow rays of light to enter the 

15 imaging system. Further, the surface at 4 can optionally be appropriately shaped to account for the 
refractive effects of the transparent block, so the shape of jthe surface 4, in general was different from 
the shape of the surface 2. The best way to keep a single point of view for the whole system is to make 
both refractive surfaces 1 and 4 as spherical surfaces with normal incidence of the principal rays 
entering and leaving from the system. At the same time, their radii of curvatures should have the 

20 specific values not introducing chromatic aberrations in the working wavelength range of the whole 
system. 

6.7. Example Seven 

FIG. 15 shows a seventh exemplary opto-mechanica! arrangement for the two mirror 
system. The secondary mirror was held in place by a transparent dome 1 made of appropriate material 
25 to which a cylinder 4 was attached. The secondary reflector was mounted on this cylindrical 
attachment. The surface of 4 was coated to prevent light from entering the cylindrical attachment. The 
shape of the dome and the shape of the cylindrical attachment were determined to match the optical 
.. arrangement of Che system. 

6.8. Example Eight 

30 FIG.'s 17, 18, and 19 show an exemplary arrangement in which the two-mirror system 

was integrated in a single optical block. Both mirrors were back reflectors that provided for good 
environment protection. Of the four surfaces of interest in the optical block, the first and the last 
surfaces were refractive ones, such that principal rays cross the surfaces under normal incidence 
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these surfaces, were handled by achromatization of the surfaces. Both refractive surfaces preferably 
have broadband multi-layer coatings to, increase image contrast, and to prevent ghost images on the 
detector surface. 

To prevent ghost images, ambient light and increase image contrast two pins (a) were 
5 introduced from both reflective surfaces and a groove (b) was appropriately introduced at the 
intermediate real image location (shown in FIG. 19). Note that these pins may actually comprise holes 
drilled* through the solid optical' block, and coated or surfaced so as to be opaque. Similarly, the 

grooves are machined into the optical block. 

• ..V ; v. ..Sc. .••:v : *qo7;-iv - * • ~ r - - 

A : relay, system is positioned after the second reflective surface of the mirror block to project 

10 the real image on the flat sensor (shown in FIG. 17 and 18). The relay system is either a standard 

camera lens with appropriate focallength or, preferably, a specially designed micro-projection lens to 

get a powerful high-resolution system. ........ > 

6.9. Example Nine 

FIG. 20 shows an exemplary arrangement that is similar to the third embodiment, in 
15 which the two mirror system comprises a primary hyperboloidal truncated reflector and a secondary 
spherical reflector/ The shapes of the 'two reflectors are different from the third exemplary 
embodiment. As in the third example,' system field of view is 220 degrees in the vertical plane, but 
image diameter is 20 mrn^ almost six times more than in the third example. Focal length is -5.5 mm, F 
number is 4, relay focal length is 28 mm, and back focal length is 42. 15 mm, The two mirror system 
20 fits as an attachment to single lens reflex or digital cameras to provide about five times higher 
resolving power than in the third example. 

6.10. Example Ten 

FIG. 21 shows an exemplary arrangement which is similar to the ninth embodiment, 
the primary difference being that the primary reflector is a 14 th order hyperboloidal surface of 
25 revolution. 

6.11. Example Eleven 

FIG. 22 shows an exemplary arrangement which is similar to the tenth embodiment, 
the primary difference being that the primary reflector is a 14 th order ellipsoidal surface of revolution. 

Accordingly, the invention also includes a novel method for designing a panoramic imaging 
30 apparatus. The commercial goal of methods for the design of a panoramic system or panoramic 
apparatus is to define a system's optical and packaging parameters, which will provide general 
specification characteristics and appropriate optical image quality. 
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We have discovered a new proprietary method for designing a super wide--ang!e panoramic 
imaging apparatus with a 360° horizontal field of view and up. to 260° vertical field of view. The 
apparatus comprises at least four elements: a two-mirror subsystem, a relay system and an image 
sensor. The method is based, on three well-known statements from , analytical geometry and optical 
5 system design related to the conic section curves and surfaces of revolution produced by these curves. 
From analytical geometry it is well-known about a specific feature of their focal points. Using conic 
surfaces as reflectors, we have two aberration free conjugate points, their geometrical foci. This means 
the two foci are perfect conjugate points. All rays from one focus pass through the other after 
reflection by the surface. Because both points lie on the optical axis, spherical aberration is absent in 
10 full. Combining two conic reflective surfaces we will keep this feature. Putting entrance and exit 
pupils of the optical system at these points will produce a single view point for ihe whole system. We 
can do this by putting a real diaphragm at one geometrical focus or by optically project hardware 
aperture of camera lens like in-US* Patent 5^760,326 to Nayar. - - " 

The second weli-known statement from geometry tells us about a stigmatic image of the entire 
15 field of view for any object positions, if we put pupils in the geometrical foci (M*M.Rusinov, 
Technical Optics, Mashinbstroenie, Leningrad, 1979, p. 250). This allowed building an anastigmatic 
panoramic imaging apparatus with a single viewpoint using a parabolic mirror, a relay lens and a 
standard camera lens. Such an apparatus is described in US Patent 5,760,826 to Nayar. A 
disadvantage of the apparatus disclosed in that patent is a field curvature of the image, which is created 
20 by the parabolic mirror. 

Due to optical system theory, to make an, image plane flat we have to correct PetzvaFs 
curvature in the optical system, i.e., to make PetzvaFs sum.zero. In a two. mirror syster^,, for example, 
we have simply to use concave and convex, mirrors with equal radii , of curvatures, (Virendra N. 
Mahajan, ; Optical Imaging and Aberrations, SPIE Press, 1998, p. 375). , This will provide image 
25 curvature .correction in third, order approximation, i.e., close to the optical axis. In super wide-angle 
systems it is not enough to correct only PetzvaFs curvature to make an image plane flat, because of 
existing field curvature of the high orders. 

Nevertheless, the present inventors have discovered that it is possible to provide a "self- 
correcting" reflection pfa scene, which is substantially free of optical aberrations of. major concern in 
30 super wide-angle systems (namely,, astigmatic effects and field curvature, effects; there are., of course 
other optical aberrations, such as coma and chromatic effects, but these are of lesser concern) in a two- 
mirror system using the optical powers of the mirrors (or reflectors), their conic constants, and the 
distance between them. We have developed a proprietary method for designing high quality, super 
wide-angle Panoramic Imaging Systems with' 360° horizontal field of view and up to about 280° 
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eliminated in the present invention by> adopting a confoca!. arrangement that also provides a 
substantially single reference viewpoint. , ^ <> 

There are two basic steps in this method:'!) analytical calculation of the initial variant 
parameters, 2) optimization of the initial variant, using optical design prdgiirhs. ff the output of the 
5 second step leaves something to beftesirfcd, the first step should be repeated' feV ^^many repetitions as 
necessary', WaclfteVe 1 the image quality ^sire^ It ; is to* be' understood that Some dpitic^l design 
softwarej : iike CODE V, ^KlAXj OSL© brother commercial programs, cain c l)e' Used on the second 
step. 

To calculate initial parameters the analytical formulae were worked out and programmed. 

10 r; In v , general, ,an r! inventive imaging #pparatu£ contains, four components^ two reflectors or 
mirrors, a, relay system and a $ensor.i- For.the specific example we. ; used .a CCD as & sensor, and a 
camera lens as a relay system. To j,starfc pur --calcu|atioiis^.the'gm^l ( specification characteristics 

should be defined in advance. They are CCD size, camera lens focal length, and field of view in the 

- "i^i, ' :ku- '.tin ?v >;ilf-/ *n fsf.xs: '.v.:' ?f^.«»- m p v;,.;*;-!/,;.* bi*:;^."»;n 

vertical object space plane. We introduce in the program the radius of the image circle on CCD-plane 
;-*ij'j. r * vr.o.q jv "i iuqiiq } ; .q w t VAd*\?.t o 

15 rim. It is equal to half-vertical CCD sizej Fcl is a camera focal length, pi is half field of view angle in 

the object space. Further, we have to introduce the camera lens position relative to the first mirror 

■ . • 7 . *-*». *<■ >! : ~ a < ?.:ic.:; jrtioqv-si" b'^fiL .*■ , *=^ r ^-lr,!:;?:,.- 1 " ^'T^'.r;:' 

vertex d21. More precisely, it is a distance from the first mirror vertex and the camera lens entrance 

* _ ' • - i * ; •'. »p«?j:.S VJ Mi i;'^d^a^>h si -_':JiS'-;:*MJi. ft*: **:.k»o .sr.*.' r:-;. • f . •. - 
pupil (positive number), and radii of curvatures at the 1 st and 2 nd mirror vertexes (both radii of 

.... ■.. " b 1 : 1 '. 1 1 -, j '. : ! J.^ 3: ;.: v u"i •'•'!'"-*' " r I : - ; ■ ■ 

curvature are positive and approximately equal to each other as a first approximation for field 

'* ':i i-J " 

20 curvature correction). It is a well-known fact that field curvature equals zero in the third order 
approximation (i.e., around optfcaPaxis or^fof moderate field of view angles) if both radir of curvature 
in two mirror systerrt are equal to each other, and one of them is ! c6hvex ami the other one is concave. 
Radii" of curvatures of the 1 st mirrbrplay 'a^l^of^'kiiding^fi^r^ actually, brie -has to introduce 
only tSvo packaging data' dllsc ahd R2sc. The list nuniber "is a linear magnfficatiori of a camera lens. 

25 It is an iteratioft parameter, < because bh^doeVnof krioV if in advarice.'Pbr the first approximation one 
can put it eqlial to zefoV^if thelcameia^ lenVwori^s 'frbhi infinity. ,J ' l * ;v 

As a result, one gets four solutions with different combinations of conic surfaces of revolution. 
One such i solution 1 ' utilizes a* firSt hypVboloiddl 'mirtor ahd a ^second ellipsoidal' mirror. As stated 
already, if the initial ; soliitiotis are found unsatisfactory fbr some reason/ one coiild ' repeat the 
30 calculations, changing radii of mirror curvatures and cameraleri^locationT'fof example. 

To get the final system parameters one has to make computer optimization of the initial 
variant, which is only a simple technical procedure apparent to one of ordinary skill in the art. 

As a result, one can. develop -different kinds of super wide-angle panoramic imaging 
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field curvature correction over the entire field of view (FIG. 2A). In the Example Four (FIG. 10) we 
have even higher level of field curvature correction, introducing in the system very small amounts of 
astigmatism (less than 5 micrometers, FIG. 1 1 A). Tangential curve has.three zeros, and non-flatness 1 
micrometer. Sagittal curve has two zeros, and less than 5 micrometers non-flatness. 

5 When attempting to 1 obtain PetzvaPs curvatures* of al] examples provided, all the values 

obtained are non-zero. This resulteortfinris the fact that it is not eaoagh to correct Petzvai ? s curvature 
in the super wide-angle system to get ! image plane flatness. Instead, the shapes and sizes of the 
primary and secondary reflectors have to be set, estimated and/or optimized to obtain a reflection that 
is substantially free of the major optical aberrations of concern to the super wide-angle system 
10 technologist. 

The present invention is not to be limited in scope by the examples and embodiments disclosed 
herein, which are intended as single illustrations of one aspect of the invention, and any which are 
functionally equivalent are within the scope of the invention. Indeed, various modifications of the 
invention, in addition to those shown and described herein, will become apparent to those skilled in the 
15 art from the foregoing description. Such modifications are intended to fall within the scope of the 
appended claims. AH patents and any publications mentioned herein are herfeby incorporated by 
reference. 
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WHAT IS CLAIMED IS: 

1. A panoramic imaging apparatus for sensing a super wide-angle field of view 

comprising: 

5 . (a) a pair of. reflectors, . including, (i) a primary reflector having a-first shape and a 

first size,and capable of .providing a firsts reflection of a super wide-angle scene, and (ii) a secondary 
reflector having a- second shape and.a second size, -which-is positioned tq ; accept the first reflection to 
.: provide a second reflection at -a first ^location, the shapes; jmd sizes, of the primary and secondary 
reflectors, and their respective; positions, are ( such that the second reflection provided at the first location 
10 is substantially free of field curvature effects and astigmatic effects; 

(b) a relay system positioned to accept the second reflection and comprising optics 
capable of substantially filtering out optical, rays other than those reflected directly by the secondary 
reflector from incoming optical rays reflected directly by the first reflector to provide a filtered 

reflection; and . . , . , . .... r j ....... . 

l5 (c) an image sensor positioned to accept the filtered reflection, in which the 

panoramic imaging apparatus captures the super wide-angle scene from a substantially single reference 
viewpoint. 

2. The panoramic imaging apparatus of claim 1 in which the relay system includes optics 
capable of relocating a reflection at a first location to a second location to provide a relocated reflection. 

20 3. The panoramic imaging apparatus of claim 1 in which the second reflection is 

substantially flat and stigmatic. 

4. The panoramic imaging apparatus of claim 1 in which the primary reflector is convex 
and the secondary reflector is concave. 

5. The panoramic imaging apparatus of claim 1 in which the primary reflector is 
25 substantially hyperboloid and the secondary reflector is substantially ellipsoid. 

6. The panoramic imaging apparatus of claim 1 in which the image sensor comprises 
photographic film. 

7. The panoramic imaging apparatus of claim 1 in which the image sensor comprises a 
CCD image sensor. 

30 g. The panoramic imaging apparatus of claim 1 in which the image sensor comprises a 

CMOS image sensor. 

9. The panoramic imaging apparatus of claim 1 in which the image sensor comprises a 
video camera. 

10. The panoramic imaging apparatus of claim 1 in which the relay system is positioned 
35 such that the primary reflector is between the secondary reflector and the relay system. 
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11. The panoramic imaging apparatus of claim 1 in which the relay system comprises a 

pinhole. 

12. The panoramic imaging apparatus of claim 1 in which the relay system comprises at 
least one lens. 

5 13. The panoramic imaging apparatus of claim 1 in which the relay system comprises at 

least one mirror. 

14. ^ THe panoramic imaging apparatus of claim 1 in which the primary reflector comprises a 
substantially hyperboloidal mirror having ^surface that substantially obeys the equation expressed in 
cylindrical coordinates, r 2 = 2R x z-{\ + k x )z 2 \ where z is the coordinate along the optical axis, r is the 

10 radial coordinate, Rj is the radius of curvature at the vertex of the conic that forms the primary reflector 
and A; is the conic constant of, the primary reflector. 

15. . * The panoramic imaging apparatus of claim I in which the primary reflector has.an axis 
of rotation, comprises a mirror truncated at a plane that is substantially perpendicular to the axis of 
rotation and further has a near focal point and a far focal point. 

15 16. The panoramic imaging apparatus of claim 1 in which the secondary reflector comprises 

a substantially ellipsoidal or spherical mirroi having a surface which 'substantially obeys the equation 
expressed in cylindrical coordinates, > 2 = 2R 2 z-(V+k 2 )z 2 \ where Vis the coordinate along the optical 
axis, r is the radial coordinate, 722 * s the ? radms 'of curvature at the 'Vertex of the conic that forms the 
secondary reflector and *2 5 s the con ^ constant of the secondary reflector. 

20 17; ' The' panoramic imaging apparatus of claim 1 in^which the secondary reflector has an 

axis of rotation, which is substantially coincident with the axis of rotation of the primary reflector, 
comprises a mirror truncated at a plane which is substantially perpendicular to the axis of rotation of the 
secondary reflector and further has a near focal point and a far fecal point. 

' ' 18. :? The panoramic imaging apparatus of claim 1* in which a position of the near focal point 
25 of the secondary reflector is substantially coincident with a position of the far focal point of the primary 
reflector. 

19. * The panoramic imaging apparatus of claim 1 in which the relay system is positioned 
aiong the axis of rotation of the primary reflector and the secondary- reflector. * 

20. The panoramic imaging apparatus of claim i in which the image sensor is positioned 
30 along the axis of rotation of the primary reflector and the secondary reflector. 

21. The panoramic imaging apparatus of claim 1 further comprising a transparent 
mechanical support, which maintains the relative positions of the primary reflector, the secondary 
reflector, the relay system and the image sensor. 
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signal processor that is coupled to the image sensor to convert the 'image signal into image signal data 
and to map the image signal data into a Cartesian coordinate system. 

23. The panoramic imaging apparatus of claim 1 in which the image signal processor further 
includes an interpolator for providing interpolated image data, such that the interpolated image data and 

5 the image signal are combined to form a digital image. 

24. /The. panoramic^ ; imaging; apparatus^ of claim^l in r which the; primary and secondary 
reflector include back -reflective, mirrored surfaces of a solid transparent optical block. „ 

-. ■ , •- \ J2Sr > '* The panoramic imaging apparatus of claim 24-in whicK the transparent block has at least 
one hole or groove. , . , o - rf: x *f- ■? i -i.vn..,. i.r 

10 26. The panoramic imaging apparatus df ~fciaim r 2S in which the at least one hole or groove 

Has in axis that d6fnbi<fes witliWoptical axis of the'pinmauy'jind setdndaiy reflectors. 

■ . .« u< * -v v.L^l-r q v zU ' j'< hh .. ^ .or ;> \* wri'u. tJ >ti\. : * . ic\ -■ • . 

27. A method of providing a substantially stigmatic reflection of a super wide-angle field of 

view captured from a substantially single reference viewpoint comprising: 

r,r, ' (a) ^•fjproyWipg.a^rim^reflectojlp^ of a super wide-angle 

15 scene^the pq^^ Reflector haying a near, focal point and a far focal point; ^ ^ 

: (b) ^ ).v providing; ar^escem 
second reflection, the secondary* reflector hayings near foeal point and a far focal pojnt; .... , . . 

• (c) { providing .a relay system,* haying r an entrance, pupil;, in which the primary 
reflector; secondary reflector and lens, are, positioned such that ? the,-fer focal point o£the primary reflector 
20 coincides substantially with>the near focaLpoint of the. secondary reflector: and the ,enjrance pupil of the 
relay system coincides substantially with the far fpci4iX)ji.9i^£th9S^n#iy: reflector. t , . , iT 

28 ^ jo The method.of^laim^T^iQ which;^ysvpf light pf . the second reflection pass through a 
hole in the.primany reflector .and. meet ; substantially at-^ primary 
reflector. 

25 : 29. ,/Fhe method, of claim 27: which r ^^err.cpmprises substantially.; filtering put optical rays 

other than those reflected directly .by the secondary ..reflector from, incoming optical, rays reflected 

directly by the first reflector to provide a filtered reflection. 

/ j *.v.;r r * - \: } !' f - ■ .• *>> >-i :iyv*K' \:>. \.-.y ■ t * „.; 

30. The method of claim . 27 which .fyrther comprises propagating the second reflection 
through the relay system to generate a relocated reflection, 

30 31. . The method of claim 29 5 which •further comprises propagating the filtered reflection 

through the relay system to generate a relocated reflection. 



BNSOOCID: <WO 0041024A1_I_> 



WO 00/41024 



PCT/USOO/00050 



34. The method of claim 27 in which the primary reflector and the secondary reflector have 
shapes and sizes that give rise to a second reflection that is substantially free of field curvature effects. 

35. A panoramic imaging apparatus for sensing a super wide-angle field of view 
comprising: 

5 (a) a pair of reflectors, including (i) a primary reflector having a first shape and a 

first size and capable of providing a first reflection of a super wide-angle scene, and (ii) a secondary 
reflector having a second shape and & second size, which is positioned to accept the first reflection to 
provide a second reflection at a first location, the shapes and sizes of the primary and secondary 
reflectors, and their respective positions, are such that the second reflection provided at the first location 
1 0 is substantially free of field curvature effects and astigmatic effects; 

(b) a relay system comprising optics capable of relocating the second reflection to a 
second location to provide a relocated reflection; and 

(c) an image . sensor positioned to accept the relocated reflection at the second 
location, in which the panoramic imaging apparatus- captures the, super wide-angle scene from a 

15 substantially single reference viewpoint. < ' ■.j'v, 

36. The panoramic imaging apparatus of claim 35 in which the relay system includes optics 
capable of substantially filtering out optical rays- other than those reflected directly by the secondary 
reflector from incoming optical rays reflected directly $y the first reflector. 

37. A method of providing a' substantially planar and stigmatic reflection of a super wide- 
20 angle field of view captured from a substantially . single, reference viewpoint comprising: 

(a) providing a primary reflector having a first shape and a first size to generate a 

; *''•'*•] ■ * 

first reflection of a super wide-angle scene; and ; - / * 

' • it * , . i , 

(b) providing a secondary reflectcr, having a second shape and a second size to 
accept the first reflection : and to generate a second reflection, in which the shapes and sizes of the 

25 primary and secondary reflectors, and their respective positions, are such as to give rise to a second 
reflection that is substantially planar and stigmatic and in which a super wide-angle field of view is 
captured from a substantially single reference viewpoint. 

38. The method of claim 37 which further comprises substantially filtering out optical rays 
other than those reflected directly by the secondary reflector from incoming optical rays reflected 

30 directly by the first reflector to provide a filtered reflection. 

39. The method of claim 38 which further comprises relaying the filtered reflection to 
another location to provide a relocated reflection. 

40. The method ..f claim 39 which further comprises sensing the relocated reflection. 
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